wafer.space Community - Hi @azonenberg , I am currently

9:08 p.m.

As I understand it, there are 3 modes for the test pattern generation:

PRBS
square
pseudo-random (scrambler)

(edited)

PRBS is pseudorandom

9:10 p.m.

but using specific standardized LFSRs in particular

9:11 p.m.

I have an FPGA-proven 10GbaseR MAC/PCS that would be straightforward to add a few resets to and get working in ASIC, the scrambler block has already been ported successfully to TS28HPC+ and used with a commercial SERDES

9:11 p.m.

I wouldn't waste time thinking about Ethernet for this stage of things though. Focus on the PHY

9:12 p.m.

the line coding and framing is a solved problem once we get the PHY working

9:12 p.m.

I could write the missing pieces in an afternoon

9:12 p.m.

if you can generate and read back PRBS7 and PRBS31 that should be sufficient

1

9:13 p.m.

We can worry about whatever comes downstream of the transceiver on a future chip

9:13 p.m.

or bolt on an experimental MAC/PCS but it wont be the focus of testing

Alright, I also have a 10GBASE-R PCS, but it hasn't been successfully taped out, so yours is safer. Is the code for it public ? Mind if I torture it ? (edited)

It is public and on azonenberg/antikernel-ipcores on github, dont have the full link handy. again, it was written for FPGA and will need a little bit of tweaking

9:14 p.m.

but i have successfully got it running at one point or other on efinix titanium, xilinx kintex-7, and xilinx ultrascale+

9:14 p.m.

i have gearboxes and 64/66b coding blocks etc that we can add to it as needed

1

Perfect.

The internal-facing interface for the MAC is axi4-stream

azonenberg

The internal-facing interface for the MAC is axi4-stream

I hate you .... (Just kidding.)

at some point i'll polish things up more and document it etc

I will go though the code and see what we want to add: does it have a MDIO interface ? (edited)

No

1

9:16 p.m.

just logic outputs for link up etc

9:16 p.m.

I wanted to add some performance counters and adjustments to it in the future, but it would probably just be a register interface

9:17 p.m.

Mind if I bolt one on then? I think we really want one. I can handle it.

Here's a capture of it running on an Efinix platform

1

9:18 p.m.

Again, let's worry about the phy first

9:18 p.m.

given the transceiver, the MAC/PCS is trivial. there are no major engineering challenges and there's existing code including but not limited to mine that we can build on

9:19 p.m.

the transceiver is the part we don't have, and that is actually hard

Yeah, but my analog skills are far from good enough to be able to design a transiver, so I will stick to what I know I can do.

Well feel free to poke at my code but i suggest you not spend a lot of time on porting at this stage

9:23 p.m.

I am planning to do some improvements here to make it more portable across a wider range of transceivers, and documenting it better, in the coming weeks/months

1

9:23 p.m.

So you might end up with a lot of merge conflicts if you mess with the code much

9:24 p.m.

especially things around clocking and resets that will be important for asic porting are going to get revamped

9:24 p.m.

it ideally will end up being something that will run on any xilinx or efinix part, or asic, with few if any rtl changes needed to integrate

The bigger question, honestly, is how things like the CRC are going to do timing wise

9:40 p.m.

in the MAC more than the PCS. the PCS is straightforward

9:40 p.m.

but the MAC was written for 28nm and smaller FPGAs

9:41 p.m.

I do not know how well it will perform on GF180 especially without the 1.8V transistors

9:41 p.m.

I think 1000baseX and PCIe gen1/2 are far more reasonable goals for GF180MCU, and 10G is probably something we should be targeting at a future 90/65/40nm open PDK

9:41 p.m.

I would be happy if someone proves me wrong and gets 10G working on 180 though

azonenberg

I am planning to do some improvements here to make it more portable across a wider range of transceivers, and documenting it better, in the coming weeks/months

Tapeout is november 9th, so unfortunalty it looks like I will take the risk. (edited)

9:45 p.m.

Do you have a top for the PCS somewhere, looking at the code I see a lot of modules but I was wondering if you had another repo where you fit them all together ?

This is purely an IP library repo

9:52 p.m.

interface/ethernet-stream/AXIS_XGEthernetMACWrapper is the top level wrapper for the Xilinx FPGA version of the block

1

9:53 p.m.

It includes an UltraScale+ GTY instance, clocks from the quad PLL, the PCS, and the MAC

9:53 p.m.

goes all the way from external differential pairs to axi streams facing the FPGA internals

9:53 p.m.

There are a lot of state machines and blocks in there that rely on FPGA power-on reset so you'll have to add external reset inputs for all of that

9:54 p.m.

You may have to increase pipeline depth for some stuff

( perfect I just to happen to have an UltraScale+ in the corner )

But if you don't have a transceiver yet, how are you going to tape this out?

9:54 p.m.

i'm confused

Yeah, not a fan of the 64b wide data path too ...

Essen

Yeah, not a fan of the 64b wide data path too ...

10GbaseR is natively a 64b block size although my MAC is mostly running double rate at 32 bit datapath

9:55 p.m.

since that's what the GTY typically gives you

9:55 p.m.

It passes timing for a -2 kintex7 with a GTX as well, i've used it there successfully

9:55 p.m.

i did have to add an extra pipeline stage there after having originally wrote it on ultrascale+

namibj is doing the transceiver, I am doing the digital side and we are targeting the chipalooza challenge

ah ok you're working together

9:56 p.m.

If you poke around there are 64/66b gearboxes under serdes/ in the same repo

1

9:57 p.m.

and block alignment

9:57 p.m.

that i've used to interface with transceivers that don't include this functionality natively

9:57 p.m.

the wrapper i mentioned above assumes your transceiver has the gearbox (edited)

9:57 p.m.

but yeah, i did not try super hard to save on area etc because i targeted this at large FPGAs where e.g. block ram is cheap

1

Right, you don't provide a gearbox. I will just bring mine.

I have a gearbox

9:58 p.m.

it's just not in the wrapper

9:58 p.m.

it's in serdes/linecode/

9:59 p.m.

Gearbox32PlusHeaderTo32 and Gearbox32To32PlusHeader are probably the ones to look at

1

10:00 p.m.

All the pieces are there, i just dont have any docs on how they fit together yet

10:00 p.m.

but i am using them in real designs across several FPGAs internally

1

Perfect, at least then I can be confident that they work.

and again they're missing the asic resets

1

10:01 p.m.

especially on the axi stream versions rather than the older legacy ones using the native interface

10:01 p.m.

some stuff is reset on the axi stream ARESET#

10:01 p.m.

but not everything

10:02 p.m.

note also that the datapath supports running at both 312.5 and 322.2625 MHz depending on if you are using the weird xilinx pll flow that gives you a 32b data word every clock

10:02 p.m.

or if you are just at 1/32 rate on the serdes and occasionally have gaps in the data due to gearboxing

1

10:02 p.m.

so there's a data-valid strobe to account for

10:02 p.m.

and the axi stream on the tx side can occasionally backpressure

1

10:02 p.m.

the rx side stream does not support backpressure and if you can't process the data as it comes in, any FIFOs are up to you as the integrator (edited)

1

For now I will try to assemble a top level with your code and get an area estimate + see how much work timing needs.

10:11 p.m.

Thanks @azonenberg

The older version of the MAC had FIFOs that needed block ram, the AXI version backpressures on the TX so I don't think there is any SRAM anywhere in the mac/pcs

10:11 p.m.

what you put on the far side of the MAC is up to you

10:12 p.m.

But yeah, timing was not designed for an old slow cmos process

10:12 p.m.

so i am curious just how bad it will be

10:12 p.m.

There is also a PRBS31 generator core under math/ in that same repo BTW

1

10:12 p.m.

that you should be able to just mux straight into the serdes at whatever bit width it runs at

10:13 p.m.

i don't currently have a PRBS checker IP you'll have to write that yourself

1

10:13 p.m.

i dont think i ever wrote PRBS7 but 31 is the more important test pattern since it more closely approximates the spectral characteristics of scrambled 10GbaseR data

10:13 p.m.

PRBS7 is better to simulate 8b10b

1

azonenberg

so i am curious just how bad it will be

I should have the results soon, maybe by tomorrow, but I don't think it will be too bad. Let's see.

10:23 p.m.

The first node we are targetting is the ihp cmos 5l, which is about ~8x faster than the gf180 7 track. But I think I will indeed be having a lot of fun with it ...

oh, if you're going for IHP it'll definitely be a lot faster lol

10:25 p.m.

i was thinking gf180mcu with 3.3/5v cells

Apparently the 9 track cells are a lot faster, but with just the 7 track closing that timing at 3.3V sounds like hell on earth

10:50 p.m.

That being said ... the plan of the challenge is to port the IP to all the nodes, and gf is defiantly on that list. So sooner or later I will have to walk though that hell ...

Lol. Well let me know how it goes, I'll see if i have time to do the cleanup of the MAC/PCS on my end

10:54 p.m.

I do also have a tri-speed 10/100/1000 baseX/SGMII MAC and 8b10b block

10:54 p.m.

which i think is likely to be easier to close timing for

1

Thanks, and in this trade I can offer the 100BASE-T/RMII MAC.

I probably have one of those around somewhere too but I don't use RMII much

12:09 a.m.

Normally I just run RGMII for everything since I have RGMII PHYs all over the place and even if i don't need gigabit it's easier than inventorying another phy and support passives

12:09 a.m.

i can just slap the same design down anywhere and use it

azonenberg

I think 1000baseX and PCIe gen1/2 are far more reasonable goals for GF180MCU, and 10G is probably something we should be targeting at a future 90/65/40nm open PDK

We're targeting Tim Edward's IHP tapeout for now; then doing the muxed e-test for W.S. Run3; then porting the SerDes from IHP to the next one in line (I think it was aimed roughly at W.S. Run4?).

azonenberg

note also that the datapath supports running at both 312.5 and 322.2625 MHz depending on if you are using the weird xilinx pll flow that gives you a 32b data word every clock

~~ngl I was thinking about 33b wide to kill the gearbox but 3¹ × 11¹ is not a good vibe to target with the MUX~~

azonenberg

i dont think i ever wrote PRBS7 but 31 is the more important test pattern since it more closely approximates the spectral characteristics of scrambled 10GbaseR data

Do you know if it's easier or harder to check than the scrambled idle test pattern they added as an optional one in the 25GBASE-R PCS?

Yeah 32:1 serialization followed by 33:32 gearboxing is the way to do it IMO

namibj

Do you know if it's easier or harder to check than the scrambled idle test pattern they added as an optional one in the 25GBASE-R PCS?

I have not looked at 25GbaseR so can't comment. I believe it is the same 64/66b line code as 10GbaseR but with FEC added on top

8:58 p.m.

PRBS7/31 are trivial LFSRs and super easy to check

And any wants regarding statistics that go beyond very plain&obvious counters?

So, ideally if you want something the most flexible and useful - i don't expect this for the first tapeout

8:59 p.m.

your receiver would have two slicers with controllable phase and voltage offset

8:59 p.m.

One always tries to be in the eye center, the other is adjustable

azonenberg

I have not looked at 25GbaseR so can't comment. I believe it is the same 64/66b line code as 10GbaseR but with FEC added on top

Yeah the FEC is only lacking in CR-S and SR-S; it only has minor additions on 10G other than that FEC.

And you can sweep phase/voltage shift and check how much of the time they disagree

9:00 p.m.

Which will let you do full BER eye scan

9:00 p.m.

This is how serdes eye scan works on e.g. xilinx transceivers, and IMO is a feature we very much want in a production grade open serdes IP

9:00 p.m.

it isn't a day-one requirement for a first tapeout but the more you can validate early on the better

9:00 p.m.

and it will make receiver margin testing much easier if we can eye-scan internally, post equalization

9:01 p.m.

we can then play with CTLE settings to see what gives the most open eye at the RX

9:01 p.m.

If done right, this eye scan is non-intrusive (the actual signal-out slicer doesn't change) and can work on arbitrary data not just a PRBS

azonenberg

your receiver would have two slicers with controllable phase and voltage offset

So, the good thing, if lucky a phase interpolator thingy falls out of the CDR design blocks; if that's easily digitally controllable I would very much attempt to give you a capture ability akin to the 10G reclocker you linked a couple days ago.

Great. Look at e.g. the xilinx GTX/GTY datasheets for an example of how you'd use this

9:02 p.m.

I have code written for the GTX, haven't done GTY yet

9:02 p.m.

Long term, I would want an APB or similar register interface on the transceiver that we can use to access eye scan as well as various configuration settings like enabling PRBS test, controlling emphasis, etc

9:02 p.m.

It probably makes sense for this to be an external module

9:03 p.m.

i.e. the transceiver has a native interface with separate ports you can drive directly from RTL, then you supply a bus wrapper with a memory map around those ports as an optional addon

9:04 p.m.

If you have a proposed feature set and RTL interface documented anywhere yet, I can give feedback I've worked with multiple vendors' FPGA SERDES as well as one ASIC SERDES so far

9:04 p.m.

So i have a decent idea of what i do and don't like

azonenberg

we can then play with CTLE settings to see what gives the most open eye at the RX

I'm not gonna attempt CTLE if I can get enough amplitude in to where I can hit the eye box with the whatever kind of clocked comparator it's gonna rely on.

well anyway, internal eye scan is something we need long term

9:10 p.m.

if it can't make the initial tapeout, we'll survive

9:10 p.m.

but it is a must-have feature for a long term professional grade transceiver

azonenberg

PRBS7 is better to simulate 8b10b

I did/do want to be able to express that spectrum for e.g. the USB-SS, DP HBR2/3 (5.4/8.1 GBaud), PCIe 1/2, SATA 3, and depending on performance possibly SAS-3 (12G NRZ 8b10b).

Yes. Which is why i suggested you implement both

9:18 p.m.

Given my PRBS31 module it should be trivial to make a PRBS7 generator, just change the length of the LFSR and tweak the tap positions

9:18 p.m.

should be like two lines of code changed

9:19 p.m.

or i can do it and fpga verify it at some point

Is loopback from an offset slicer if the clock recovery is sufficiently independent to stay locked an acceptable substitute for an internal eye scanner? At least if e.g. area limitations stand in the way of an actual second slicer & demuxer package?

@Essen

Other resources you might want to consider looking into are:

LiteEth

https://github.com/corundum/corundum

GitHub - corundum/corundum: Open source FPGA-based NIC and platform...

Open source FPGA-based NIC and platform for in-network compute - corundum/corundum

1

namibj

Is loopback from an offset slicer if the clock recovery is sufficiently independent to stay locked an acceptable substitute for an internal eye scanner? At least if e.g. area limitations stand in the way of an actual second slicer & demuxer package?

What do you mean loopback from an offset slicer? can you draw a dtapath block diagram or something?

azonenberg

What do you mean loopback from an offset slicer? can you draw a dtapath block diagram or something?

I'll draw you a diagram in bit once I'm at the office.

Tim 'mithro' Ansell

@Essen

Other resources you might want to consider looking into are:

LiteEth

https://github.com/corundum/corundum

Yup, alex is going an amazing job there.

On another topic, I think I will be not be using your RTL, just as inspiration for getting some early area utilization number for now @azonenberg (and won't be actually porting it for now). (edited)

OK, doesnt matter to me one way or the other. I do plan to polish it up to an ASIC-friendly format at some point but it was not on my near term agenda

1

azonenberg

What do you mean loopback from an offset slicer? can you draw a dtapath block diagram or something?

2:45 p.m.

Sorry forgot to send it yesterday.

Reason is in part because depending on how the clock recovery works, it might pretty much need an adjustment dial for the slicer phase and depending on how DC offsets in the input buffer/amplifier/etc. work, slicer level might also need to be adjusted vertically anyways, which together would make the main slicer already capable of the eye pattern sampling. Together with if clock recovery can run on the edges independently of the slicer-proper, I.e., without depending on the data recovery part of the whole CDR, this would allow using the phase lock of normal CDR while telling the data slicer to sample deliberately not in the center of the eye. And if that outcome could be exported by feeding the "recovered" data to a TX serializer that's slaved to the same recovered clock, this would AFAIK allow something resembling a "SAR-ADC sampling oscilloscope". Notably the transmitted bitstream may lack sufficient transition density for easy CDR, but as it's slaved to the RX recovered clock, I'd assume a far far smaller PLL bandwidth for the CDR (together with a lock of frequency being hard-locked to the test signal sent to the DUT) to suffice and ideally thus allow dealing with such poor patterns. I mean it might need to be captured with an ADC and processed in configurable or even programmable compute, but oh well...